1. Field
Embodiments presented herein provide techniques for analyzing streams of computer data. More specifically, techniques are disclosed for analyzing and learning patterns of behavior from monitoring computer networks.
2. Description of the Related Art
Intrusion detection systems (IDS) monitor network or system activities for malicious activities or violations and produce reports to a management console. Many current intrusion detection systems are signature-based. That is, an IDS may be configured with signatures to detect malicious or unwanted activity. As known, an attack signature is a sequence of computer activities (or alterations of those activities) corresponding to a known attack, such as towards a vulnerability in an operating system or application. An attack signature may also be a bit pattern in a network packet corresponding to a known vulnerability. An IDS may use attack signatures through passive protocol analysis (e.g., using a “sniffer” in promiscuous mode) or signature analysis (e.g., interpreting a specific series of packets or data within that packet to a known pattern of attack). For example, an IDS may be configured with an attack signature that detects a particular virus in an email. The signature may contain information about subject field text associated with previous e-mails that have contained the virus or attachment filenames in the past. With the signature, the IDS can compare the subject of each e-mail with subjects contained in the signature and also attachments with known suspicious filenames.
However, the signature-based approach raises several concerns. For instance, although an IDS may possibly detect alterations to a particular attack, the alterations typically must be defined in the signature to do so. Similarly, because attack signatures are predefined, the IDS is susceptible to new attacks that have not yet been observed. That is, someone generally has to observe a particular attack pattern or instance of an exploit before defining a signature. As a result, an IDS may be unable to detect so-called “zero-day” attacks (i.e., attacks that exploit a previously unknown vulnerability in a system or application). As the number of systems is increasingly being breached by zero-day attacks, the signature-based approach in intrusion detection systems becomes of limited usefulness. In addition, attackers may use code morphing techniques to subvert attack signatures, making the attacks difficult to detect.